Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Content: Add Fire dataset #234

Merged
merged 24 commits into from
Jun 21, 2023
Merged

Content: Add Fire dataset #234

merged 24 commits into from
Jun 21, 2023

Conversation

danielfdsilva
Copy link
Collaborator

@danielfdsilva danielfdsilva commented Apr 3, 2023

Adding content

@mccabete This is the first draft of the Fire Dataset. We can start adding content directly here


Content checklist

Check the Content Documentation for more details.

Any content

  • Cover image with proper attribution (when relevant) is present.
  • Cover images do not contain any text (they're used as a background and text may not be readable).
  • All images are not too big/heavy. As a general rule of thumb, they should not weigh more than 500KB, or be larger than 2000px. See the media guide for more info.
  • Content was added using the available MDX Blocks.

Datasets

@netlify
Copy link

netlify bot commented Apr 3, 2023

Deploy Preview for visex ready!

Name Link
🔨 Latest commit 9799f02
🔍 Latest deploy log https://app.netlify.com/sites/visex/deploys/64931dddbf48ac0008652bed
😎 Deploy Preview https://deploy-preview-234--visex.netlify.app/
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@mccabete
Copy link
Contributor

Hey @danielfdsilva, I am just now sitting down to add some content to this.

  1. I watched the walkthroughs. Great stuff, super helpful.
  2. Is this still the place I should put content? I see this got merged.

@danielfdsilva
Copy link
Collaborator Author

Hey @mccabete.

Yes, this branch (content/fire) would be where you add the content. What got merged was "deploy v0.8.0" that references this PR and therefore shows up as a note. This PR is still in Draft state.

Let us know if you have questions

@mccabete
Copy link
Contributor

mccabete commented May 17, 2023

@danielfdsilva I put a PR with some text in. It's probably not the final text, but it's something.

One question that has already come up showing this to EIS fire folks, is a question of how much, if any, customization of the exploration map we can do? I know we touched on this during the virtual meeting, but I need a reminder of where that line is so I can report back.

There are two cool things we are envisioning for the exploration map, are they possible?

The first is that each of the fires would show the full progression of the fire (ie multiple fire perimeters per fire, like this).

The second would be to go from the perimeter of a fire to a set of analyses/ notebooks/ visualizations, tied to that area. This seems like it could be an extension of the work implemented in the "analysis" , only instead of the user drawing a custom area of interest, the fire selected would define the area of interest, and we would bring up a wider range of automatically generated plots.

@danielfdsilva
Copy link
Collaborator Author

@mccabete Those are interesting features but currently not possible to achieve.

To show multiple fire perimeters would mean to show multiple time entries (days) simultaneously, the current dashboard assumption is that only one time entry is shown at any given time.

Having a more customized notebook analysis entry point (i.e. going to the notebook with current exploration parameters like area) is something we have in our backlog but was not yet implemented. cc @j08lue

Let me know when and if I should have a look at the PR you created.

@mccabete
Copy link
Contributor

Hey @danielfdsilva,

Thanks for getting back to me! Feel free to look at the PR. Could you also give me a "weeks vs months vs year vs 5 years" sense of when those changes could be implemented?

For my first point, ( fire evolution as shown by multi polygons), would it make a difference if we organized the data differently? I am specifically trying to envision a VEDA frontend home for the 'public.eis_fire_lf_perimeter_nrt collection that @ranchodeluxe and @smohiudd just whipped up. The real "value added" from that collection is that you can see the fire progression.

@j08lue
Copy link
Contributor

j08lue commented May 22, 2023

@mccabete and @danielfdsilva

We could consider visualization of vector feature time series as a potential feature for the Dashboard. This type of visualization is quite common, not only to fire mapping but also coastlines and other features evolving over time, I think, so this is a generic enough need.

The technical complexity depends a lot on where we would want to implement this. Integrating visualization of time ranges of vector data, including a legend showing the time in color (optionally plus additional stats such as area or length - in the case of line features), might further complicate the already challenging design of the combined Exploration and Analysis page.

Creating a separate kind of view only for vector data would perhaps be easier and faster - but also here the challenging part is how to integrate such a feature logically into the Dashboard, keeping it generic.

Alternatively, we could publish Jupyter notebooks that generate these kinds of visualizations on embedded maps and encourage users to run these notebooks when they want that type of visualization. That is the cheapest option, I would say.

@mccabete, who would you like to present this data visualization to? Who would be looking at these graphs and how would they want to interact with the plots? Would a notebook be an option for that kind of audience - with some static data included that they could view just in a static render of the notebook and the option to pick other fires / time periods when they execute the notebook (in our Jupyter, in MyBinder, etc)?

@mccabete
Copy link
Contributor

@j08lue -- notebooks would be a great starting point. I think we were envisioning our audience as "fire-science invested public". And that these interactive visuals would be a way to connect our fire tracking work to our pre-fire weather risk work and post-fire risk work. I am slightly worried by what you mean by "static". I could see this being fine or not depending on the implementation. Ideally, we'd want visuals that can adapt to the most recent large fires on the landscape and don't need someone updating a curated list of example fires -- although that may be where we start.

What we really want are both a "timeseries of vectors" like you mention, and also "selecting a polygon givens you access to visualizations/ notebooks specific to that polygon" -- I think this is also a generic need (I could see people using this for census tracks, states, ecoregions etc). Is this also a need you think could be met with the notebook solution you outline above?

@j08lue
Copy link
Contributor

j08lue commented May 26, 2023

Ideally, we'd want visuals that can adapt to the most recent large fires on the landscape and don't need someone updating a curated list of example fires

That would be what I would call "dynamic" in contrast to static data sourcing - the data can be changed by the user or automatically when new becomes available. A notebook file would not allow for dynamic data loading. But as soon as a user executed the notebook, they could change/update the data.

Executing Jupyter notebooks does not sound like something a member of a "fire-science invested public" would do, though, even if we made it very simple.

although that may be where we start

Here is my suggestion. We are currently having active discussions (that also @ashiklom is contributing to), about how to integrate use-case-specific visualizations into the VEDA Dashboard. I expect this support evolve in the future, but it will be a while.

To start with what we have, I suggest that we create a proof-of-concept Jupyter notebook and include it in our examples, that generates such a fire evolution visualization, loading data from our vector features API. Then scientists like you would have a tool to generate this visualization, supported by the VEDA platform. You could also consider turning that into a Streamlit app and link to that, which would be a shortcut to generating custom interactive visualizations that non-technical users can use.

We will continue to strategize the support of use-case-specific visualizations and use your case as an example, if we start to make experiments or so.

How does that sound? Would you be able to create a notebook and submit a PR to https://github.com/NASA-IMPACT/veda-docs? Else we could open an issue in that repo and try to find someone to support.

@j08lue
Copy link
Contributor

j08lue commented May 26, 2023

selecting a polygon givens you access to visualizations/ notebooks specific to that polygon

If I understand correctly, you mean that any polygon on the map (e.g. fire line or predefined boundaries) could be used directly as AOI for calculating statistics like those in the current Analytics (and more).

That is an interesting idea and we just in the context of the Dashboard Exploration+Analysis design discussed different mechanisms for users to define/pick their AOI. This could be one of them. I'll pass this idea on to the designers (cc @ricardoduplos) for inspiration.

Otherwise, let us put this second idea aside for now, because it is technically a very different thing, and focus on the custom visualization.

@mccabete
Copy link
Contributor

mccabete commented Jun 5, 2023

@j08lue we can certainly build some custom notebooks for some of our visualization goals. Here is an example notebook focusing on a specific fire, but that has the visuals we would be interested in executing on a per-fire basis.

@j08lue j08lue mentioned this pull request Jun 21, 2023
7 tasks
@j08lue
Copy link
Contributor

j08lue commented Jun 21, 2023

I see we discussed a lot of different things in the context of this PR. I'll create new issues for those, where applicable.

Let us focus here on getting this content merged.

Can you tell what is left to do before this dataset addition can be merged, @mccabete, now that I merged the other branch you had open into this PR?

This is the dataset in preview: https://deploy-preview-234--visex.netlify.app/eis/datasets/fire

@j08lue
Copy link
Contributor

j08lue commented Jun 21, 2023

Unfortunately, the dataset fails to load:
image

@j08lue
Copy link
Contributor

j08lue commented Jun 21, 2023

I guess the veda-ui submodule needs an update, too? I'll try whether a rebase helps.

@j08lue
Copy link
Contributor

j08lue commented Jun 21, 2023

Unfortunately still the same error:

image

@smohiudd
Copy link
Contributor

smohiudd commented Jun 21, 2023

@j08lue looks like the collection name that the front end is tying to access eis_fire_perimeter has changed so it's returning a 404 error.

@mccabete what are the collections that we should be using for the veda frontend?

@ranchodeluxe
Copy link
Contributor

@mccabete and @smohiudd : the previous name now maps to this collection: https://firenrt.delta-backend.com/collections/public.eis_fire_snapshot_perimeter_nrt so maybe we can just make that change?

Copy link
Contributor

@ranchodeluxe ranchodeluxe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pre-approval 😉

@smohiudd
Copy link
Contributor

Last thing I need to do is update the STAC collection id to eis_fire_snapshot_perimeter_nrt and it should work

@smohiudd smohiudd marked this pull request as ready for review June 21, 2023 18:29
@smohiudd smohiudd merged commit 0a371d5 into develop Jun 21, 2023
@smohiudd smohiudd deleted the content/fire branch June 21, 2023 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants